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Information  contained  in  this  report  was  presented  at  the  Air  Force 
Systems  Command  Junior  Officers  Scientific  and  Engineering  Symposium, 
23-25  August  1966,  Aerospace  Medical  Divisions  USAF  School  of 
Aerospace  Medicine. 
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ABSTRACT 


This  paper  describes  an  Electronic  Systems  Division  effort  to  adapt  the 
AFSCM  375  series  and  AFSCM  310  series  management  techniques  to  computer 
program  acquisition.  These  techniques  are  unique  in  that  they  provide  the  first 
standardized  management  approach  to  computer  program  design  and  development. 

A  complete  process  of  management  procedures  that  has  been  developed  for 
management  of  computer  programs  during  definition  and  acquisition  phases  is 
discussed.  Particular  emphasis  is  placed  on  an  extensive  change  package  to 
AFSCM  375-1  Configuration  Management  During  Definition  and  Acquisition  Phases 
and  a  package  of  computer  program  data  items  for  insertion  in  AFSCA/^AFLCM  310-1 
Data  Management  proposed  by  Electronic  Systems  Division  of  AFSC.  The  concepts 
of  uniform  specifications,  baseline  management,  change  control,  specification 
maintenance  and  accounting  and  standard  data  items  as  they  apply  to  computer 
programs  are  discussed  and  the  impact  of  applying  these  techniques  to  System 
Programs  is  analyzed. 


INTRODUCTION 


In  the  development  of  large  scale  computer  based  systems,  the 
application  of  management  techniques  to  the  design  and  development  of  computer 
programs  has  lagged  far  behind  the  management  of  hardware  acquisition.  While 
the  concepts  of  uniform  specifications,  baseline  management,  change  control, 
etc.,  described  in  the  AFSC  375  series  documents  were  titled  "System 
Management  Techniques11  ,  these  techniques  had  been  devised  to  manage 
hardware  acquisition  and  did  not  cope  with  the  peculiarities  of  computer  programs. 
A  number  of  reasons  account  for  the  reluctance  of  the  Air  Force  to  manage 
computer  program  design  and  development.  Primarily,  the  computer  program  was 
an  elusive  and  intangible  object.  It  could  not  readily  be  seen  or  felt  and  thus 
it  could  not  be  easily  described.  The  computer  program  was  not  hardware,  nor  was 
it  data.  It  was  easy  to  change  the  computer  program  to  correct  design  deficiencies 
and  to  avoid  redesign  of  hardware,  but  it  was  soon  realized  that  uncontrolled 
changes  created  total  confusion.  Thus,  the  computer  program  defied  description. 
The  mystery  associated  with  computer  programs  created  numerous  management 
problems.  Detailed  technical  requirements  were  not  specified  prior  to  computer 
program  design.  Interfaces  between  computer  programs  and  hardware  or 
personnel  were  not  specified  and  expensive  incompatibilities  were  often  designed 
into  systems.  Sufficient  documentation  was  not  provided  for  the  computer 
program  and  often  the  user  was  never  able  to  operate  the  system  effectively. 

The  lack  of  management  techniques  for  computer  program  development  was 
creating  expensive  overruns  and  costly  systems  that  failed  to  satisfy  user 
requirements. 

RELATIONSHIP  TO  375  SERIES  DIRECTIVES 


The  existing  Systems  Command  375  Series  management  procedures 
establish  uniform  requirements  for  system,  equipment,  and  facility  contract  end 


item  specifications.  Techniques  for  establishing  baseline  management, 
implementing  change  control  and  conducting  design  reviews  are  also  provided. 
Unfortunately,  none  of  these  techniques  address  computer  programs.  The 
computer  program,  being  an  integral  part  of  many  systems,  naturally  had  to  be 
considered  in  this  uniform  management  approach  to  provide  system  compatibility. 

An  examination  of  Air  Force  Systems  Command  375  management  techniques 
indicated  that  many  of  these  so-called  11  Systems  Management  Techniques" 
could  be  adapted  to  the  management  of  computer  program  design  and  development. 

An  effort  was  thus  initiated  at  Electronic  Systems  Division  to  adapt  the  375  series 
procedures  to  the  management  of  computer  programs.  The  fundamental  concept 
of  this  approach  was  to  define  computer  programs,  i.e.,  a  sequential  list  of 
digital  computer  instructions  on  magnetic  tape,  punched  cards,  etc.,  as  a 
deliverable  contract  end  item.  The  Computer  Program  Contract  End  Item 
(CPCEI)  is  similar  in  many  ways  to  an  equipment  CEI  as  defined  in  AFSCM  375-1 
Configuration  Management  During  Definition  and  Acquisition  Phase  (I).  It  is  a 
deliverable  item  that  is  formally  accepted  by  the  procuring  agency.  It  is  the 
prime  level  for  management  control  and  accountability  and  for  preparing 
technical  manuals.  The  CPCEI  is  described  by  a  desigr/requirements 
specification  in  the  same  manner  as  equipment  CEIs  are  described  by  a  Part  I 
CEI  Specification  (I). 

Once  the  computer  program  was  defined  as  a  CPCEI,  it  was  determined 
that  many  of  the  concepts  of  existing  375  series  manuals  and  other  research  and 
development  directives  could  be  adapted  to  management  of  the  CPCEI.  Specifically, 
the  concepts  of  uniform  specifications,  baseline  management,  change  control, 
specification  maintenance,  design  reviews  and  a  test  program  were  deemed 
necessary  for  adequate  management  of  computer  program  development. 
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UNIFORM  SPECIFICATIONS 


The  uniform  specification  program  (I)  defines  the  system  specification  and 
the  CEI  specification  that  comprise  that  system.  The  system  specification 
presents  the  design/performance  requirements  that  the  system  must  meet.  Thus, 
it  provides  the  basis  for  development  of  both  hardware  CEI  specifications  and 
computer  program  CEI  specifications.  The  CPCEI  specification  (2)  is  written  in 
two  parts  in  the  same  manner  as  hardware  CEI  specifications,  as  described  in 
AFSCM  375-1.  Part  I  of  the  CPCEI  specification  provides  the  desigrVperformance 
requirements  and  Category  I  qualification  test  requirements  for  the  CPCEI.  It 
consists  of  a  detailed  description,  in  operational  and  mathematical  language, 
of  the  functions  to  be  performed  by  the  CPCEI.  The  Part  I  specification  is  the 
basis  for  design  and  development  of  the  CPCEI  and  contains  the  requirements 
against  which  the  CPCEI  is  tested.  Part  II  of  the  CPCEI  specification  is  a  detailed 
technical  description  of  the  CPCEI  as  delivered.  It  contains  a  technically 
oriented  description  of  the  functions,  structure,  data  base  organization,  etc., 
of  the  CPCEI  including  detailed  flow-charts  and  source  statement/machine 
language  listings.  Following  its  completion,  the  Part  II  specification 
constitutes  a  reference  to  assist  the  user  in  diagnosing  troubles,  designing 
modifications  and  implementing  changes.  As  such,  its  technical  accuracy  and 
completeness  must  be  assured  prior  to  its  acceptance  by  the  Air  Force. 

BASELINE  MANAGEMENT 

A  baseline  is  defined  as:  “An  approved  and  defined  point  of  departure 
for  control  of  future  changes  in  system  or  computer  prograrr/equipment  performance 
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and  design.  Each  baseline  is  technically  defined  by  a  specification  and 
typically,  a  system  would  have  three  unique  baselines,  as  shown  in  Figure  I: 
the  Program  Requirements  Baseline  defined  by  the  system  specification;  the 
Design  Requirements  Baseline  defined  by  the  Part  II  CE1  specification;  and  the 
Product  Configuration  Baseline  defined  by  the  Part  II  CEI  specification.  Baseline 
management,  then,  is  the  establishment  of  accurately  defined  baselines  and  the 
implementation  of  procedures  to  control  changes  to  these  baselines  and  insure  that 
the  system,  as  delivered,  reflects  all  approved  changes. 

CHANGE  CONTROL 

Change  control  and  specification  maintenance  form  the  cornerstones  of 
baseline  management.  Change  control  establishes  systematic  procedures  for 
proposing  changes  to  an  end  item  or  established  baseline  and  for  evaluating  these 
changes  prior  to  approval,  while  specification  maintenance  establishes  detailed 
procedures  for  updating  the  baselined  specifications  to  reflect  the  approved  changes. 
Thus,  the  baselines  are  meticulously  controlled  to  insure  that  they  do,  in  fact, 
accurately  represent  the  system  and  its  end  items. 

DESIGN  INTEGRITY 


Design  reviews  and  inspections  (1)  provide  the  procuring  agency  and  the 
contractor  scheduled  pauses  in  the  design  process  for  review  of  the  design  effort. 
The  Preliminary  Design  Review  (PDR)  and  Critical  Design  Review  (CDR)  provide 
two  reviews  of  the  design  process;  the  PDR  at  an  early  stage  and  the  CDR  when  the 
detailed  design  is  complete.  The  First  Article  Configuration  Inspection  (FACI) 
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ESTABLISHMENT  OF  BASELINES 
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Figure  I 


provides  an  audit  of  the  technical  documentation  and  the  qualified  computer 
program  to  insure  that  the  documentation  accurately  describes  the  CPCEI. 

The  requirement  for  testing  the  computer  program  contract  end  item  is 
satisfied  by  a  qualification  test  program  based  on  the  Category  I  tests  discussed 
in  AFR  80-14.  Each  CPCEI  qualification  test  program  is  conducted  to  insure  that  the 
CPCEI  satisfies  the  design  requirements  specified.  The  qualified  CPCEIs  and  CEIs 
are  then  evaluated  via  a  Category  II  test  program  that  tests  the  total  system. 

A  TYPICAL  COMPUTER  PROGRAM  ACQUISITION 


An  examination  of  the  System  Life  Cycle  of  a  typical  computer  based 
command  and  control  system  will  describe  the  application  of  the  management 
techniques  to  computer  program  development.  The  typical  system,  as  described  in 
Figure  2,  consists  of  a  number  of  system  segments  (I)  of  which  only  the  information 
processing  segment  is  germane  to  this  discussion.  The  information  processing 
system  is  comprised  of  equipment  contract  end  items  that  constitute  the  computer 
and  its  peripheral  equipment  and  one  or  more  computer  program  contract  end  items. 
In  the  example  chosen,  two  CPCEIs  are  identified:  the  Air  Defense  Computer 
Program  and  the  Utility  Computer  Program  Package.  Other  CPCEIs  could  be 
Maintenance  and  Diagnostic  Computer  Programs,  System  Exercising  and  Simulation 
Computer  Programs,  Etc. 

Early  in  the  Definition  Phase  (DOD  Directive  3200.9,  I  July  1965)  the 
system  to  be  developed  is  defined  by  the  system  specification  in  terms  of  system 
design/performance  requirements.  The  approved  system  specification  technically 
defines  the  first  baseline,  the  Program  Requirements  Baseline,  and  identifies  the 
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TYPICAL  COMPUTER  BASED  SYSTEM 
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Figure  2 


system  segments  that  comprise  the  system.  As  a  product  of  the  Definition  Phase, 
the  definition  contractor  provides  a  contract  end  item  detailed  specification 
(Part  I)  for  each  contract  end  item  within  the  system  segment  for  which  he  is 
responsible.  The  approved  contract  end  item  specification  (Part  I)  defines  the 
second  baseline  for  each  end  item,  the  Design  Requirements  Baseline.  The  CPCE1 
specification  (Part  1)  as  shown  in  Figure  3  fulfills  two  primary  functions.  It 
defines  the  performance  and  design  requirements  for  the  computer  program  CEI  and 
it  identifies  the  test  requirements  that  will  form  the  basis  for  qualification  testing 
of  the  CPCE1  later  in  the  life  cycle.  Note  that  the  system  specification  is 
structured  in  the  same  way  and  performs  the  same  functions  at  the  system  level. 
Within  the  performance  requirements  are  included  the  interface  requirements  of 
the  CPCE1  with  other  equipment  and  computer  program  end  items.  A  brief  outline 
of  the  CPCEI  Part  I  specification  is  provided  in  Appendix  1.  The  Part  I  specification 
is  structured  functionally  corresponding  to  the  major  functions  to  be  performed  by 
the  CPCEI.  The  Air  Defense  Computer  Program  for  example,  would  have  functions 
such  as  aircraft  tracking,  aircraft  identification,  weapon  control,  etc.,  while  the 
Utility  Package  would  have  such  functions  as  assembler,  compiler,  tape/memory 
dump,  etc.  Note  that  as  the  detailed  design  is  developed,  the  CPCEI  will  be 
structured  into  computer  program  components  that  satisfy  the  design  requirements 
of  the  Part  I  CPCEI  specification.  But  a  one  to  one  relationship  between  computer 
program  components  and  functions  does  not  always  exist.  Any  one  computer  program 
component  may  satisfy  all,  none,  or  some  of  the  design  requirements  of  a  particular 
function  described  in  the  Part  I  CPCEI  Specification.  An  example  of  this  will 
be  given  later  in  the  discussion  of  the  Part  II  CPCEI  Specification. 

At  the  start  of  the  Acquisition  Phase,  detailed  design  of  the  contract  end 
items  commences.  The  functions  of  the  Part  I  Specification  are  allocated  to  the 
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RELATIONSHIP  OF  PART  I  CPCEI  SPECIFICATIONS  TO  CPCEI  DESIGN  AND  TESTING 
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DEVELOPMENT  OF  PART  1 1  CPCEI  SPEC  TEST  PLAN  &  PROCEEDURES 


computer  program  components;  the  functional  flow  within  the  CPCEI  is  developed; 
the  data  base  structure  is  designed,  other  design  activities  take  place.  The 
documentation  of  the  detailed  design  from  start  to  finish,  i.e.,  from  allocation 
of  functions  to  complete  machine  listings,  forms  the  CPCEI  specification  Part  II. 

The  Part  II  specification  is  a  detailed  technical  description  of  the  computer 
program  contract  end  item  and  evolves  as  the  detailed  design  progresses  from 
functional  flow  diagrams,  to  high  level  flow  charts,  to  detailed  flow  charts, 
to  coding  of  instructions  and,  finally,  to  complete  machine  listings.  A 
number  of  parallel  efforts  are  conducted  as  the  detail  design  develops.  The 
Category  I  qualification  test  program  is  being  planned  and  drafts  of  the  test  plan 
and  associated  test  procedures  are  being  written  for  SPO  approval.  The  relation¬ 
ship  of  the  detailed  design  and  the  Category  I  test  program  to  the  Part  1  CPCEI 
specification  is  shown  in  Figure  3.  Supporting  documentation  such  as  user's 
manuals,  positional  handbooks,  simulator  guides,  etc.  are  being  prepared  in 
draft  form  as  various  details  of  the  design  become  rigid. 

The  first  management  milestone  in  the  Acquisition  Phase  is  the  preliminary 
design  review  (PDR),  usually  held  within  60  days  after  the  contract  award  for  the 
Acquisition  Phase.  The  PDR  may  be  held  for  one  or  more  CPCEIs  and/or  CEls  as 
required,  e.g.,  a  PDR  could  be  held  for  the  whole  information  processing  system 
segment.  At  the  PDR,  the  design  approach  of  the  CPCEI  is  reviewed  with 
particular  emphasis  on  the  various  interface  requirements  of  the  CPCEI.  The 
Part  I  Specification  and  those  portions  of  the  Part  II  specification  that  describe  the 
structure  and  overall  functions  of  the  CPCEI  form  the  basis  for  the  PDR.  Specifically, 
the  following  information  would  be  available  for  review  at  a  PDR:  computer 
program  functional  flowcharts;  storage  allocation  charts;  control  functional 
description,  data  base  organization  and  structure.  Particular  emphasis  is  placed 
on  the  interface  requirements  of  the  CPCEI  with  other  CPCEIs  and  hardware  CEls. 
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A  review  of  word  lengths,  message  formats,  available  computer  storage,  timing, 
etc.  is  conducted  to  insure  that  the  requirements  of  the  Part  I  CPCEI  Specification 
and  the  System  Specification  have  been  met.  At  the  PDR,  interfaces  between  the 
CPCEI  and  equipment  CEls  should  be  sufficiently  defined  so  as  to  preclude  future 
definition  at  a  lower  level  of  detail.  It  will  be  expected,  however,  that  inter¬ 
faces  with  other  CPCEIs  will  require  subsequent  definition  at  a  lower  level 
of  detail . 

As  the  design  of  the  CPCEI  progresses,  the  individual  computer  program 
components  are  assigned  to  groups  of  individuals  for  design,  flowcharting, 
coding,  etc.  Design  and  development  of  the  computer  program  components 
proceeds  in  a  parallel  manner  from  this  point  on  until  formal  qualification 
testing.  During  this  design  process,  *he  requirements  of  the  Part  I  specification 
which  are  function  oriented  are  translated  into  the  actual  CPCEI  which  is 
structured  into  computer  program  components. 

The  relationship  of  the  components  of  a  CPCEI  to  the  functions  identified 
in  the  Part  I  CPCEI  specification  is  shown  in  Figure  4.  As  the  design  of  each 
component  proceeds  to  the  detailed  flowchart  level,  a  critical  design  review  is 
held  for  that  component.  In  this  manner,  the  CDR  for  a  CPCEI  is  performed 
incrementally  by  computer  program  components.  Due  to  the  varying  complexity 
of  the  parallel  design  efforts  for  computer  program  components,  it  would  be 
unreasonable  to  delay  all  of  the  components  being  developed  to  hold  one  CDR  for 
a  computer  program  contract  end  item. 

The  critical  design  review  for  a  CPCEI  (2)  is  a  technical  review  of  the 
design  integrity  of  the  CPCEI.  The  CDR  is  accomplished  incrementally  by 
computer  program  components  when  the  design  is  essentially  complete,  i.e.,  after 
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preparation  flowcharts  but  prior  to  coding  of  the  component.  This  does  not 
preclude  coding  portions  of  complex  CPCEIs  if  necessary  to  meet  schedules.  In 
addition,  any  coding  required  to  demonstrate  design  integrity,  such  as  testing  of 
algorithms,  may  be  accomplished  prior  to  CDR.  At  CDR,  the  completed  sections 
of  the  Part  II  CPCEI  specification  are  reviewed  along  with  supporting  analytical 
data,  test  data,  etc.  The  compatibility  of  the  CPCEI  design  with  the  requirements 
of  the  Part  I  specification  is  established  at  CDR.  11  Inter11  interfaces  with  other 
CPCEIs  and  "intra"  interfaces  between  computer  program  components  are 
examined  to  insure  compatibility.  The  design  integrity  is  established  by  review 
of  analytical  and  test  data  in  the  form  of  logic  designs,  algorithms,  storage 
allocations  and  associated  methodology.  Immediately  following  the  CDR, 
coding  of  individual  components  takes  place  and  the  process  of  checkout  and 
testing  of  the  components  begins. 

The  Category  I  test  program  demonstrates  that  the  CPCEI  as  produced 
satisfies  the  desigr\/performance  requirements  of  the  Part  I  CPCEI  specification. 

The  Category  I  test  program  must  be  designed  to  insure  that  all  of  the  functional 
requirements,  as  translated  into  computer  program  components,  are  tested  and 
that  nothinggets  lost  in  the  translation.  The  Category  I  test  program  is  subdivided 
into  two  major  classes  of  tests:  Preliminary  Qualification  Tests  (PQT)  and  Formal 
Qualification  Tests  (FQT).  The  Preliminary  Qualification  Tests  are  designed  to 
verify  the  performance  of  individual  components  prior  to  an  integrated  formal 
qualification  of  the  complete  CPCEI.  The  PQT  is  conducted  incrementally  by 
components  in  the  same  manner  as  the  CDR.  Figure  5  depicts  the 
relationship  between  CDR  and  the  Category  I  test  program.  The  crosshatched 
blocks  indicate  coding  of  individual  computer  program  components.  The  CDR, 
coding  and  PQT  are  conducted  sequentially  on  an  incremental  basis,  component 
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Figure  5 


by  component.  The  PQT  is  modular  and  a  "building  block"  effect  occurs  as 
testing  progresses.  As  each  computer  program  component  is  added  and  each  PQT 
conducted  increased  confidence  develops  in  the  CPCEI  being  tested.  At  the 
conclusion  of  PQT,  all  of  the  computer  program  components  have  been  integrated 
and  tested  and  the  CPCEI  Is  ready  for  formal  qualification  and  acceptance. 
Qualification  testing  of  an  operational  CPCEI  such  as  the  Air  Defense  Computer 
Program  requires  extensive  use  of  simulation  techniques.  The  use  of  these 
techniques  is  dictated  by  the  high  cost  of  providing  overhead  computer  facilities 
or  by  the  unavailability  af  new  computers  undergoing  a  parallel  design  and 
development  effort.  Although  PQT  will  make  maximum  use  af  simulation  techniques, 
the  Formal  Qualification  Tests  af  an  operational  CPCEI  will  require  live  inputs, 
live  outputs  and  operationally  configured  equipment.  A  prerequisite,  then,  af 
FQT  is  usually  the  installation  and  checkout  af  the  CPCEI  in  an  operationally 
configured  computer  at  the  Category  II  test  site.  The  exception  would  be  in  the 
case  af  a  support  CPCEI  such  as  a  utility  package  that  would  nat  require  live 
inputs,  e.g.,  radar  data,  and  cauld  be  fully  qualified  at  the  contractor's  facility. 

Ta  provide  reliable  data  during  FQT,  the  CPCEI  installation  requires  fully 
qualified,  installed  and  checked  out  equipment  CEIs.  The  first  opportunity  far 
FQT  will  occur  at  the  Category  II  test  site  after  qualified  CEIs,  that  have 
successfully  passed  First  Article  Configuration  Inspection,  have  been  installed 
and  checked  aut  and  an  operationally  configured  system  exists.  Subsequently, 
installation  and  checkout  af  the  CPCEI  occurs  and  FQT  begins.  The  conclusion 
af  FQT  signals  the  end  of  the  Category  I  test  program.  The  CPCEI  has  been 
fully  qualified  and  all  af  the  requirements  of  the  Part  I  specification  have  been 
satisfied.  An  exception  to  this  would  be  those  requirements  af  the  Part  I 
specification  that  cauld  only  be  demonstrated  by  a  Category  II  system  test. 

After  a  successful  FQT,  the  CPCEI  has  been  fully  Integrated  inta  the  system 
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and  is  ready  for  acceptance.  The  design  and  development  of  the  CPCEI  is 
essentially  complete  except  for  those  residual  errors  discovered  during  system 
testing. 

With  the  design  and  testing  of  the  CPCEI  completed,  the  CPCEI 
Specification  Part  II  is  available  for  review.  The  Part  II  specification  as  the 
detailed  technical  description  of  the  CPCEI  contains  the  technical  discussion  of 
the  CPCEI  and  all  of  the  computer  program  components  that  comprise  it.  It  will 
accompany  the  CPCEI  to  each  installation  or  site  and  function  as  the  primary 
document  for  "Maintenance"  of  the  CPCEI.  As  said  before,  the  technical 
accuracy  and  completeness  of  the  Part  II  specification  must  be  determined  prior 
to  its  acceptance  by  the  Air  Force.  The  First  Article  Configuration  Inspection 
(FACI)  provides  the  vehicle  for  the  required  review  of  the  Part  II  specification. 

The  FACI  is  an  audit  of  the  Part  II  CPCEI  specification  and  the  CPCEI  as 
delivered.  The  result  of  FACI  is  the  acceptance  of  the  CPCEI  specification 
(Part  II)  as  the  technical  definition  of  the  third  and  last  baseline,  the  Product 
Configuration  Baseline.  Subsequent  to  FACI,  the  configuration  of  the  CPCEI  is 
essentially  controlled  at  the  machine  instruction  level  so  that  the  exact  configuration 
of  the  CPCEI  is  available  for  Category  II  system  testing. 

At  the  conclusion  of  FACI,  formal  acceptance  of  the  CPCEI  takes  place. 

Air  Force  acceptance  of  the  CPCEI  is  based  on  the  successful  completion  of  the 
Category  I  Test  Program  and  the  FACI,  but  it  does  not  relieve  the  contractor  from 
meeting  the  requirements  in  the  system  specification.  After  acceptance,  the  Air 
Force,  with  contractor  support,  conducts  an  extensive  Category  II  system  test 
program.  The  objectives  of  the  Category  II  tests  are  to  demonstrate  that  the  system 
will  satisfy  the  system  perfonmance/design  requirements  of  Section  4  "Quality 
Assurance"  in  the  System  Specification. 
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CHANGE  CONTROL  AND  SPECIFICATION  MAINTENANCE 


Throughout  the  design  and  development  process  described  above,  baseline 
management  is  required  to  retain  effective  control  of  the  process*  As  stated 
before,  change  control  and  specification  maintenance  are  the  primary  tools  of 
baseline  management.  They  describe  systematic  procedures  for  proposing, 
approving  and  implementing  changes  to  an  established  baseline  and  associated 
specifications.  Although  the  level  of  the  change  may  vary  from  a  system 
requirements  change  to  a  CPCE1  instruction  change,  the  procedures  are  essentially 
the  same.  The  proposed  change  is  submitted  in  preliminary  form  to  an  Air  Force 
Configuration  Control  Board  (I)  who  approves  or  disapproves  the  proposed  change. 
Once  approved,  the  CPCEI  change  is  developed,  coded  and  tested  and  the  change 
and  appropriate  specification  change  notices  (SCNs)  are  forwarded  to  the  CCB  for 
formal  approval.  Subsequently,  the  change  is  installed  in  the  CPCEI  and  the  SCNs 
update  the  appropriate  specifications.  Figure  6  depicts  the  ECP/SCN  process. 

While  change  control  is  required  for  effective  system  management,  excessive 
control,  particularly  at  the  level  of  computer  program  instructions,  i.e.,  the 
product  configuration  baseline,  will  restrict  the  contractors  design  effort.  This  is 
particularly  true  early  in  the  testing  and  11  debugging"  of  the  CPCEI  when  numerous 
errors  in  coding  are  discovered  and  corrections  made.  By  holding  the  FAC  I 
immediately  prior  to  Category  II  system  testing,  only  the  Design  Requirements 
Baseline  is  established  during  the  design  and  Category  I  testing  of  the  CPCEI. 

Since  the  number  of  errors  detected  throughout  the  life  of  a  computer  program  is 
probably  best  approximated  by  an  exponential  function  approaching  zero,  many  of 
these  errors  will  have  been  detected  prior  to  FAC1.  To  further  prevent  overcontrol 
of  the  contractors  design  effort,  two  classes  of  changes  are  defined,  the  class  I 
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change  and  the  class  II  change  (2,3).  A  class  II  change  is  one  which  the 
contractor  may  effect  without  prior  approval  by  the  configuration  control  board 
and  at  no  additional  cost  to  the  procuring  agency,  e.g.,  changes  to  correct 
editorial  errors,  computer  program  errors,  etc.  A  class  I  change  which  always 
requires  prior  approval  is  any  change  not  falling  within  Class  II  as  defined  above, 
i.e.,  a  change  that  effects  operational  capability  as  specified  in  the  baselined 
Part  I  CPCEI  specification,  contract  price  or  schedule,  interfacing  CEIs,  etc. 

The  Class  I  changes  are  processed  as  described  above,  but  the  class  II  change  is 
submitted  for  approval  of  its  classification  only  after  the  change  is  installed  but 
prior  to  release  of  the  associated  SCNs. 

DOCUMENTATION  REQUIREMENTS 


As  the  design  process  for  computer  programs  evolved  and  management 
techniques  were  applied,  specific  documentation  requirements  were  uncovered. 

These  documentation  requirements  were  compatible  with  the  Air  Force  Data 
Management  Program  (4)  and  an  effort  was  made  to  develop  standard  data  items 
(4)  to  describe  these  requirements.  This  effort  resulted  in  data  items  (documentation) 
falling  in  four  categories:  configuration  management,  handbooks,  personnel 
subsystem  and  testing.  The  configuration  management  items  describe  the  format 
and  minimum  content  for  both  Part  I  and  Part  II  of  the  CPCEI  specification.  In 
addition,  various  forms  for  change  proposals,  specification  change  notices, 
specification  indexes  and  specification  accounting  forms  are  described.  In  the 
handbook  category,  a  number  of  manuals  and  handbooks  are  identified.  The 
Positional  Handbook  for  example,  describes  all  of  the  functions  and  actions  to  be 
performed  by  a  console  operator  in  a  computer  based  system.  On  the  other  hand, 
the  Users  Manual  outlines  the  procedures  for  operating  a  particular  CPCEI.  In 
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the  personnel  subsystem  area,  a  variety  of  documents  exist  that  relate  the 
operating  personnel  to  the  system.  These  documents  describe  various  aspects  of 
the  personnel  interface  with  the  equipment  and  computer  programs  from  early  in 
the  design  stage  to  the  training  of  operators  for  the  system.  In  the  testing  area, 
the  documentation  requirements  for  the  Category  I  test  plan,  test  procedures  and 
test  report  identify  the  minimum  content  for  these  documents.  A  partial  listing 
of  ESD  computer  program  documentation  is  shown  in  Table  I. 

SUMMARY 

The  techniques  described  above  have  been  documented  by  the  Electronic 
Systems  Division  in  an  Exhibit  (2)  to  be  used  in  conjunction  with  AFSCM  375-1. 
Basically,  the  exhibit  is  a  supplement  to  AFSCM  375-1  and  it  contains  detailed 
instructions  for  applying  the  techniques  described  in  this  paper.  Although  these 
procedures  were  developed  in  conjunction  with  the  AFSC  375  manuals,  the 
concepts  can  be  readily  applied  to  the  management  of  computer  programs  in 
general.  Additionally,  ESD  has  created  a  package  of  unique  Forms  9  that 
supplement  the  data  items  provided  in  AFLCA^AFSCM  310— I  "Management  of 
Contractor  Reports  and  Data."  The  exhibit  and  the  Forms  9  have  been  forwarded  to 
Air  Force  Systems  Command  for  inclusion  in  future  revisions  to  the  basic  manuals. 

Currently,  ESD  is  selectively  applying  the  exhibit  and  the  Forms  9  to  all 
new  procurements  involving  computer  programs.  Although  experience  is  limited 
since  all  of  the  contracts  are  still  in  the  early  stages,  a  number  of  Preliminary 
Design  Reviews  have  been  held.  These  reviews  have  discovered  and  resolved 
interface  problems  between  equipments  and  computer  programs  that  would 
otherwise  have  gone  undetected.  It  is  felt  that  the  application  of  these 
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management  techniques  to  computer  based  systems  will  provide  a  first  step  in 
eliminating  or  reducing  those  problems  discussed  earlier.  The  identification  of 
computer  programs  as  contract  end  items  and  the  use  of  uniform  specifications  will 
insure  that  computer  program  requirements  are  identified  and  that  the  proper 
emphasis  is  placed  on  computer  programs  early  in  the  design  process.  With  proper 
emphasis  on  the  computer  programs,  more  realistic  trade-offs  can  be  conducted 
between  equipments  and  computer  programs. 

The  application  of  baselinemanagement  to  CPCEfs  will  insure  that  when  a 
change  in  requirements  occur,  both  the  reason  for  the  change  and  the  impact  of 
the  change  is  evaluated.  Baseline  management,  if  properly  applied,  will  insure 
that  the  interfaces  between  CPCEIs  and  equipments  or  personnel  are  given  proper 
consideration  before  a  change  is  implemented.  The  use  of  design  reviews  and  the 
modular  Preliminary  Qualification  Tests  provide  the  procuring  agency  a  method 
of  evaluating  the  design  of  the  CPCEI  while  gaining  confidence  in  its  performance. 

In  addition,  potential  incompatibilities  between  equipments  and  computer  programs 
should  be  identified  earlier  in  the  development  cycle  and  resolved.  The  result 
should  be  smoother  integrating  of  the  numerous  end  items  into  an  operational 
system.  It  is  hoped  that  the  availability  of  Standard  Forms  9  to  provide  documentation 
will  eliminate  the  situation  where  a  computer  based  system  about  to  go  operational, 
discovers  that  documentation  for  the  computer  program  is  non-existent  or  deficient. 
The  addition  of  these  management  techniques  to  the  existing  375  series  procedures 
will  provide  a  true  systems  management  program  and  better  equip  Air  Force 
Systems  Command  to  acquire  computer  based  systems. 
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